Skip to content

Reuse guarded immutable Overview deliveries for continuation - #132

Open
advancewhl wants to merge 4 commits into
aoci-spec:mainfrom
advancewhl:whloong/p0-overview-continuation-main-port-pr
Open

Reuse guarded immutable Overview deliveries for continuation#132
advancewhl wants to merge 4 commits into
aoci-spec:mainfrom
advancewhl:whloong/p0-overview-continuation-main-port-pr

Conversation

@advancewhl

@advancewhl advancewhl commented Oct 16, 2026 •
edited
Loading

Copy link
Copy Markdown

What changes and why

Large Whole-Index continuations used to repeat full governance evaluation and rebuild the formal body. The preserved rc19 baseline took 499.447 s for six chunks, with continuations around 83 s.

This change prepares one validated immutable Overview delivery and reuses it for guarded continuation. Each continuation checks current Safe Inventory and governed raw bytes without repeating full governance or rebuilding the body. The initial request still performs the full assessment. V2 cursors are Host-bound, authenticated, replayable, and bounded by count, bytes, and lifetime. Expiry, eviction, foreign Host, or restart fails closed. Reconstructable V1 continuation remains supported. Database-enabled and unsupported snapshots retain live governance.

Affected public contracts

  • No public contract changes
  • MCP tool name, input schema, or response structure
  • CLI command, flag, exit code, or --json shape
  • spec/public contract text or machine dictionary (internal/machinecontract)
  • Index format, Baseline, receipt, or transaction identity derivation

Compatibility: canonical cognition body and chunk receipts remain v1; only the opaque continuation cursor is versioned. V2 state is in-memory. Lost or expired V2 sessions require a fresh request; V1 reconstruction remains available. The entire MCP response envelope is not byte-identical.

Verification

  • make fast (Go 1.26.9)
  • make full
  • python3 scripts/blackbox/mcp_conformance.py — 53 checks
  • python3 scripts/blackbox/mcp_scenarios.py — 75 scenarios
  • python3 scripts/blackbox/mcp_lifecycle.py
  • Other: go test ./internal/mcptools -count=1; git diff --check; Verify, aggregate Check, and Guide (aligned; next_action=none)

Proven runtime behavior

Real local Codex MCP activation completed a 251-Entry delivery with overview_delivery.chunk_tokens=4000: 10 chunks, first chunk about 42.5 s, continuation median about 2.73 s, and maximum about 3.01 s. No continuation returned to the historical ~80 s range. Focused work-counter regressions show valid continuation avoids repeated full governance and body construction.

Not proven / still separate

  • Cold start remains expensive; this change does not optimize it.
  • Semantic Attestation was partial (4/10); strict cognition verification did not pass.
  • The Host-safe 4000-token setting is a deployment observation and does not establish that every Host limit is solved.
  • The frozen performance fixture is not production-state acceptance.

Operating system impact

The MCP path was exercised on Linux. make fast also built the supported Windows target; no OS-specific behavior is intended.

Migration and recovery

No persistent delivery state is added. V2 cursor expiry, eviction, foreign Host, or restart requires a fresh delivery. V1 cursors retain their live-governance reconstruction path. No data migration is needed.

Cognition

  • Managed objects changed; the AOCI governance cycle is complete (Maintain → update_entry → Verify/Check/Guide), with code and index changes in this branch
  • No managed objects changed

@dayebishouji dayebishouji mentioned this pull request Oct 16, 2026
Open
@alkor2000

alkor2000 commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Deferring this to the 0.2.0 line rather than rc22: it changes the Overview response and cursor structure, which the conformance suite and the upgrade axis pin, and rc22 is a fix-only line that promotes to 0.1.0. I'll review it there; the measurements in the description are the right argument.

alkor2000 pushed a commit that referenced this pull request Oct 26, 2026
@advancewhl @claude t
feat(overview): reuse guarded immutable Overview deliveries for conti…
9d4b757 
…nuation (#132)

Large Whole-Index continuations repeated the full governance assessment
and rebuilt the formal body on every chunk. The initial request now
prepares one validated immutable delivery (framed body, canonical
sequence, Challenge, chunk spans) and continuation reuses it behind an
input guard: effective configuration and raw config, Baseline and
Curation bytes, the composite CognitionSet identity, Safe Inventory
membership and status, the rules identity and blockers, and the raw
SHA-256 and size of every index and observe source. Any drift fails
closed with overview_snapshot_changed and a fresh request at Chunk 1.

V2 cursors are Host-session-bound, HMAC-authenticated, replayable, and
bounded by count, bytes, idle and lifetime; expiry, eviction, a foreign
Host, or a process restart fails closed. The reconstructable v1 cursor
and its live-governance path remain. Database-enabled repositories, a
pending scope change, and source warnings keep the live path. The
attestation schema boundary now names object_identity before the
semantic scorer runs, and the Ledger records the proof events.

Ported from the contributor's branch onto the 0.2.0 line, excluding the
governance assets, which the line re-authors.

Closes #132

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alkor2000 pushed a commit that referenced this pull request Oct 26, 2026
t @claude
fix(overview): serve a delivery the store cannot retain the live way
2b204a3 
The reused-delivery store has a retained-payload budget (64 MiB, counted
conservatively at eight times the body and content plus 2 KiB per Entry),
so an index of a few megabytes could not be inserted and the initial
Overview answered overview_session_capacity_exceeded instead of the body.
A delivery the store cannot hold, or a proof it cannot retain, now falls
back to the original live-governance path: the v1 cursor stays and
continuation re-runs governance as it did before, with the same verdict.
serveVolumeDelivery signals that case with a nil result.

The schema_accepted Ledger event is appended from the loaded configuration
instead of a second config load; docs/overview-delivery.md describes the
reused delivery, its guard, the v2 cursor's failure modes, the live-path
cases, and _meta.aoci_work; the spec names the fallback; the Unreleased
section records #132 with its author. Governance re-authors the sixteen
Entries the port and this change touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alkor2000 pushed a commit that referenced this pull request Oct 26, 2026
t @claude
chore(ladder): record the PR #132 closure gates on the 0.2.0 line
6dbcb9a 
make fast, make verify (conformance 53/53, scenarios 75, lifecycle 92 PASS,
axis-self 80/80), and longrun-refresh ran green on 2b204a3 with
build/aoci-132; the closure, change, and nightly cadences take today's
marks, release cadences keep rc22 until the next cut.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@alkor2000

alkor2000 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Landed on the 0.2.0 line, branch v0.2.0-line, as 9d4b757 with you as author (9d4b757), followed by 2b204a3 with the adaptations below. It is not in a release yet; the first 0.2.0 candidate will carry it.

What I changed on top:

  • A delivery the store cannot retain no longer fails the request. The retained-payload accounting is eight times the body and content plus 2 KiB per Entry, so an index of a few megabytes (about 10k Entries) exceeded the 64 MiB budget and the first Overview answered overview_session_capacity_exceeded instead of Chunk 1. Such a delivery, and a proof the store cannot retain, now take the original live-governance path with the v1 cursor; serveVolumeDelivery signals it with a nil result and two tests pin it.
  • The schema_accepted Ledger event reads the loaded configuration instead of loading it a second time.
  • docs/overview-delivery.md describes the reused delivery, its guard, the v2: failure modes, the live-path cases, and _meta.aoci_work; the spec names the fallback; the Unreleased section credits you.

Measured here on the repository's own 512-Entry index (6 chunks, one persistent session): rc22 0.61 s per call, 4.27 s for the chain plus the proof; this build 0.36 s for Chunk 1, 0.10 s per continuation, 0.14 s for the proof, 1.00 s in all. Your 251-Entry / 4000-token numbers point the same way at a larger scale.

Two things I kept as you had them and want to say out loud: _meta.aoci_work stays on the wire as a documented diagnostic, because it is what lets a suite prove which path served a call; and a lost v2: token always restarts at Chunk 1 rather than being reconstructed, which is the contract the spec now states. Thanks for the port and for the measurements; they were the argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@advancewhl @alkor2000